不能不有所顾忌

have to think twice (before doing sth.); be unable to over come certain misgivings